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(54) PANORAMA IMAGE SYNTHESIS DEVICE 

(57)Abstract: 

PROBLEM TO BE SOLVED: To provide the panorama 
image synthesis device in which plural images are 
synthesized in an excellent way, regardless of simple 
configuration. 

SOLUTION: An image picked up by an electronic camera 
307 is copied to an HD of a computer body 301 via a 
general-purpose interface 306 and displayed on a display 
device 302. Furthermore, the synthesizer discriminates 
whether an original is an original image mainly consisting of 
characters or a usual natural image after a corresponding 
point extract processing of the image relating to the image 
synthesis is finished and an overlap range of the image is 
obtained. Based on the result of discrimination, a way of 
seamless processing is changed to carry out the synthesis 
processing. 
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w®<D<y)i>-7izm-?%wm±Ttf'5-(>} t v isoi 
kxz-xz-zkwj x-&*ixm.?K-£*\, CftSro.— if 

tfltT, ^>Kt7l 8 0 2tc^-r<t-5tcf£l/^cV(D* 
-M— 5 X{a«%^*H*TSfc-£fr-e3 «fc 3 ic&o T 

xb*sj:3fcWKUiMXSttT«jj*snso ft, c 

f4<i>fcl,\, SW4, •7^XfO^>f-i'>^WX 
XWmttD r a g LTgtLfc{i[«%, tt«Hft<Dlftffi£ 
t«^T, $KIM.T*jftf* (*-r--&JjK5&JI) © 

fc<fc«<D« £<£o t*-/^— 5 -v xltsi^ 

a g^Kfeft© AN DMIS^jSttTJIS. 3 fc«>, fc^fc 
[0 0 7 6] W, ±15^*— FOV»'ftl©»&tefeV 

a-3v»TSfcSE"r a J: 3 (c->-a bx&a©^ t> **** 
a g-r*£tt©akfrrfc»K cnt4«t**rc\ 

S©j*fF-c?&3©T\ a.— »f©£fil4/h^o £fc, -fe 
54--h£-fi£cQi:£, Sifted rag LTfdVfcV^© 

[0 0 7 7]^;:, 0 1 775M02 2 4#ILOO, 0 
2 37}M0 3 KD7xn—1-v-hK'&-DX*mffi<DBM 

[0 0 7 8] S-f, Wfa^rtOBfli-r— ^fca:'*? 

a - * 'n n tf--r a m<ommc-o ^ x mm? % . 

[0 0 7 9] 02 3!i, t?*^7rtOItr-^^3 
> tf a - £ rt^3 fcf— f % RKcfr 3 #13© 7 a— ^ -V - 
M?fe-5 0 ft, 02 3©7D-f+- Mcfci^T, 

m*xoa^m*)i l tmm7 ! -^mm'>^7-^5 o i tf^© 

[0 0 8 0] IW^C&l^T, £f\ 3lf— ^g^a^ 

^»boT^S*»if3^**iJWf-rS (SlOOO)offl 
StfUta-sTV^fc&tf, t^Xf'yyS 1 0 0 9^ 

ann^itfe, %rmt>-?x^mfini£x<DX7-v?s i 
o o i^«ra*jtJ6*. 

[oo8i] u\z—mft<Dtpx\ s-r— bobm* 
T-zt*tiKttm\.rzi%mtim*wm-tz> (sioo 
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e>, £©H{g!x-*tf*^7^Xx-*frH5fr#¥iJIST 
(S 1 0 0 2) o ^T^X-r-^T'&tttxif, m 

a©xf7 7 p s i o o 4{csas^riii6i. 0 s/t, *-<f 

W h (J PEG^T I F F^F) fc^wr-YX 
t*-**^**-* (S 1 00 3) o ^LT, Sa&WKto 
ofce>7 7'l';l'*-l'X8 5 a&t/8 5 b &M#r-f£>o 
[0 0 8 2] K8 7 aStf8 7 bfcH^T 

10 (S 1 0 0 4) o ;V^vj®<i!T*&t>*§-&{i. ffi 

#©iS#-r-*t LT»f 3 (S 1 0 0 8) o JlffctfJ 
tci40 8T*©-r— ^fif- X;H 1 0 8 tcHW©^— 
Z I D*ttt*TB*U ^©7=-* I D^^a^ir- 

x;n 10 oics^-r^o 

[0 0 8 3] ^S^-rm&Xdb&ttlts 
©^©X^-Xtf^T^iiScig^frH 3 fr^x y * 
(S 1 0 0 5) o CtilZm8<Dii?ufr—7>V%: 
ftifoT^ot, ^l/-7lDO*-KID8 9atf, 
■Hfc©*— K I D 8 9 a tmVfr£5fr*3.%>£tK& 

20 t>TfTfc>tl3o 

[0084] »jfcrs^-y#*^fc*a\ X;v-x 
;&f&£-f3 (s i oo6) o cn(4, -hzupT— x;v 

1 1 0 OtcfffclC^;!/— XI D 1 1 0 2%1SU X;l/ 
-X« 110 6, f^figB^F 1107, X;l/-X^-TXi 
l l o^Mt§. X;V— X^^TXi l l Ofcti 'W 

^mm" tEtsn, Hi«<D«tt««*o-t- k i d 

8 9 a*^J6e.nf>o 

[0 0 8 5] LT, XL<r»^/ z 7'^m'^rf~^KmM<D 
=r-Z I D^WJT, fif-r;H 1 0 8tC^L, 
30 f-?IDl 105tfgf5 (S 1 007) o 

[0 0 8 6] IXlOXf-y^S 1 000*^S1008 
O— »©«HS«r3t:--r«lBfl»^Tte«LTfT5o ±x 

fc t © co 4> t\ ^ y 9 vit<D x;i/- x*Mt e. n fc *^ 

3A^^x>y^L (S 1 0 0 9) , tL#6>tlTfetl 
If, ^V7VjS^jS^IS5 1 7tC«fcf3, X^-Xrt© 
B«*fflv^»ifi-rs^/7VB«^(£«Hl*ff3 (S 

i o i o) o *ittu*cft»i:T«yi*«T-rso 

[0 0 8 7] tuiex-r yXS 1 0 1 OK&tfZ^ 

[0088] 024i±, /w?-?mm£f&wi ! m<D-7u— 

h%^-rt©T*fe§ 0 f^, 02 4©7D- ^-ir — 

[0 0 8 9] |W|0fC*3V>T, ^/7Vi«M¥a5 1 
7(4, ff, X;l/-Xrt©B«^2t!t*\ 2fe«fc'?^l/^ 
Mfx-ynS (S 1 2 0 0) o 2 ft© t £14, ^xE 

-r*7;i/*-h-&fi!t«rateA* (s i 202) o 2fe<t 

0^V^tt4, «izE-r**- h-&fi85aaKX5 (S 1 2 
so 01) . ilOXf'^S 120 1, SI 2 0 2T*©^ft 
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mfr*3-3.-yf?2> (S 1 2 0 3, S 1 2 0 4) . C© 

"r*"fe5*-h^jas*fT-3/£« (s 1 20 5) , «a 

[0 0 9 0] MIEX^-y^S 12 0 1 iC&tfZ? 

[009 1] E25ti, t-hM®I©7n-f+- 
h^-TtCDTfe^o ^2 507n- Hc*3 

[0 0 9 2] [WlSfC^V^T, /V7 Vjli^-g-fig^m 5 1 

7(*. 3-— vt)m'<®%.rz®.sMm<omm*&m 
(s i 3 o i) o #tc, -a-r*»*Sja*ao»s 

§ (s i 302) o Si, ^y^mmtvrmmt^t 

fO;l/-;l/tLt, «/M 0 %> i^5 0%*-^-7 

n5%J£TF£?£i63i:, 13 1 7tC^-r<fc5tC. ^•f*— 
^Vf-fZWmit® 1 7©;£iB« 15 0 1 ©$*Si£x 
DTI 5 0 4&Cj5VffEBi:£:£>o £;fc, =5^7 
LTl/-»*Rni6tt©**IEH«:feH«l 5 0 2©(6H6xU 
71 50 5IC^Ti!ii:45. 4\ ill'JTl 5 0 4 
©x^r^cSS^-TyM 5 0 3lC7jk-?#-<OVl$± 
B<D)ls—Mc'ft? SSHBXU7 1 5 0 5OliJ7*O 
■9--^S5H 1 5 0 6 &cjjVf x»J T^fcttjCfrrSj^tf i&3 

[0 0 9 3] 7D-^-hCII,i:, tutex-r^S 
1 3 0 2T\ fiU:©*— 

(si 303),«, c^wjiSjiajfliaiasopiffltcov^ 

f&j£©»*<mj£ffi (N» fi(±ft»£5****WrL (S 1 
3 0 4) , 3rS«WTOi:#tt. e»TJi 
otf S c ftOT, -b 5 b ^JsRJflS'N 

Jttr. ffiJMJ: 5 *R£ 
iQS'siity (S 1 3 0 5) o COM^7^-^M 

«^««M*fT3 (S 1 3 0 6) o C©ffiMfflt»»SB"r 
[0 0 9 4] MIB^f-^^S 1 2 0 2tc*5tt?>7 
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[0 0 9 5] E2 6li> 7tl/*—h&l8Xm<07U—?- 

[0096] lajBtcfc^T, >v^-*m&$imf&5 1 

7», S-f, v^>4fKH«JE*ff5tf (S 1 60 
1 ) , iintitfiffiT.^-yT'S 1 3 0 2 fciQSttH— Tfc 

[0097] mz. 4®tti&&mm9mzft5o 

10 -b^JQl©!^ ttatt2«KRSLTV^OT?, 
#^.p»ti^{LLSB8^t±, li^i iHB#2*\ ±T. T 

±\ fefe, fefe©4at)T?$>§ 0 ^CX\ C©40©« 

cne.©jaa*v x-r^rs 1 6 o 2^e,s 1 609s 

[0 0 9 8] ^LT, RlT^ffi (N 

fl) «±©*fjiSjS**ia -e* fc t> ©#& 3 x >y * 
n (si6io). tL-ofe&ij-ntf'bs^-h^ 
20 jsftia'xjttyo iL^nti *©tfT. fi^-su^ 

0«feJ:V^O*)(OffiiMflS-e**it5 (S 1 6 1 

«4 0©5"6©— 0^£ttK**t4"rf£*\ 09*. tf* KM 
«*#SULT»^fc^«;fcJ:Sfc*#ttA/T*Vvr, IE 

S 1 6 1 1T\ gfe7-f y h LTV>3t>© S(U 

^;i/3^*t>wi#-»t>©) £ra#?f 3 «t 5 &c-r 3 <, 
[0099] xrvzfs 1611 ©aaatfifeSfcs fc, % 
30 ©-^figA^^-^saa cs 1 6 1 2) % wm-Sfitmm 
cs 1 6 1 3) s\tmtst>\ cn«Buiex^-yys 1 3 
05. s 1 3 0 6 tm-mmT*&*). mmir&m?%o 

[0 10 0] HtifBXx-v7 P S 1 2 0 5lCt3lf^ 

« *- h -&fiSt$aa© # t (s a a, h ra— t 5 e 

[0 10 1] i2 7ti, -feS^-— h-a-fiSt5Q.a©7a— ^ 
+ — b*7jVrt>©T&5o ^ 0 2 7©7a-f-V-h 

5 1 7A^©ffl : iIib{ / F^T3<J:5^^oTi/>^ 0 

40 [0102] raatcfevT, /v vv@j^fig#is5 1 

■1MB*«l»-rs (S 1 7 0 1) o ^LTV<y^>yi5 

H^Rje-r** (s 1 7 0 2) „ ccT'©iBHti, 

i:^§ 0 j;oT, *—h-GrfflDim<DBZ<DWB 

[0 10 3] Xf-y-fS 1 7 0 2©5aa7b^t>5t, & 
©#JSj£«lffl«yi (S 1 7 0 3) , -S&L'^t-ZWfe 

so mm cs 1 7 0 4) , iBfi^ftffla cs 1 7 0 5) 



(9) 



«flB¥ 9-9 3 4 3 1 



15 

[0104] at, wftSjsaafflafco^TOKw^-r 
co i o 5] s 1 8*#saLT»*Sj£«iiHjagw> 

[0 10 6] CCTH1 8tts ^ttaHDB©£i:fe 

# ^ t % It, 2 «tflD-&jfK%firiHl^Ji 0 ig*tf J: v^OT'SQ, 

[0 10 7] *f % m&V$<D)l>-)UC<D-otK). "rZ/Zf 
U— h*H3£-r*«H2 0 0 5 t±, $9 0 %«f 1 0 %© 

tffiFficrsRnnttottHtv^ct^ »ioo%, « 

5 0%©iSB2 0 0 6fC&^£tt3„ IIHi!4'©7^7'V 
- b SjgffiB 2 00 5OxU7*>f), If ^«<3f 5£ffiJW 

*t-:':/u— hB*2 0 0 3 h\^TWK)&to C©-r> 
hiS^ 2 0 0 3%, It— ^®5ffl 2 0 0 4 ±KgV> 

t, co^tHWi/jNfc** 

C 5%, 2 0 0 4 ±* 1 HJS-fO-rP. LT 

*V m3tfflJWTT*ntf. J tO*-f>MB|± (x, y) 
t (x' , y' ) 4M^©^7i:LTffi»-r§c 

[0108] j^o«yi*i«»*&saiij«Mo«B5i:*a 

[0 10 9] H2 8H\ ^JS^ft&tB5aiI<D7a-^-V- 
h^-T 3c 122 8<D7D- ^-v— hlcls 

[oi io] raBtefe^T* ;V7-7liM¥i5 l 

7t±. £i\ xv^uimstaft-rs cs 1 90 

R3ef*«H2 0 0 5fr5>, i-yy^ffiW:^^ 
-fVb^rS-r (S 1 9 0 2) 0 ^LT, *©#-r>h«« 
fefttf, *CD#-/>h3^6K«±niiiJRt , O0)e}B"e]B 
«*«l»?Hil,T->:/U--hiB«J:-rs (S 1 9 0 3) o 
[0 1 1 1] ^©;}Wh©ffiBfre>, 

©■9— *«H*Kje-r* (S 1 9 0 4) o ^LT, +)— 
*-®B*©iH&fc, 7->7U-MIi«*fite-&*rta\ 18 

(S 1 9 0 5) o 

[oi 12] sk. c<Dmft<o-&%mt>\ tn^x-com 

'Nlfr^fr^x^L (S 1 906), 
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%«&-r3 (S 1 9 0 7) o UJi%"9--^-KH^Tfcl» 

[0 113] ?-LT, -9— ^HEH^T%"9— f-Lfc*^ 
x>y*L (S1908) „ *©«»#«> 6 ft fc*/.Hltf 

fcJtRLTWBfT* (S 1 9 0 9) o m««LJ:0/jN* 

Offitft^VbOli (x, y) fi/NiA^i6e» 

10 nfc^vhoasw (x' . y ' ) zom'bmvm 
i> (s 1 9 1 o) o 
[0114] &L±*Ty-7u-hwtiffim£mc*tvr 

f^(S1911), f&TLtziE>ttJ&&<DV7>htt><D£ 
T©«'JMi*»&*©¥*9«*#»* eft*— »t^;Wi 
tLTSJTfS (S 1 9 1 2) „ J^±T«^ffljffi«ia 

[0115] ^tc> fcjfi^^-^Rjeaassfco^TBt 

H^-T^o «|!*2fti:Lfci;£t;: ( 2 ttfi*±©-&jS<D« 

20 T<t^) , *©-rft«, x, y73(Rj©Mji, mm. RXf 

(x, y) , (x' , y' ) ttWT©J:3li:aHt*. 

[0 116] 

[&1] 

Mjf co.8 sineYx>j 

[zn(cos8'X + sin8'.y-Ax) "\ 
m(-sin 6 -x + cosB • 7 - Ay) ) 

l<:t\ enzmmKKDmm.n, a xso'a yiiMit, 

mti{g^*jjVr o it>T'<7^-^A, B. CMD4 
(DttfcjtmOimWX'te. tt&M (x, y) , (x' , 

y' ) (DmKDMzmmLrcc ctittm>b&m&zm^ 
40 [011 7] -r&t>-£> 

e=2 [ < (Ax + By + C) -x' } 2+ { (-Bx 
+ A y + D) -y' } 2 ] -»m i n 

[0 118] 
[&2] 



50 
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a E /aD-(^^M-(5;x)5^«D-(^y)-o 



[0 119] dCT% 
PI = Z x2 + Z y2 
P2 = Z x 
P3 =Z y 

P4 = Z x x' +Zyy' 
P5 = Z x y' -Z x' y 
P6 =Z x' 
P7 =Z y' 

P8 =n (MJS&VW) 

[0 12 0] 
[»3] 

4 P*Ps + PiPi ~ P*P» 
„2 . ^1 „ „ 



Pl + A 'Aft 

P*Pf ~ P2PT+ PsP* 

p\+p\-pip% 



c P*-P* A -P* B 
Pt 

D _ Pi-PiA+PjB 
P, 



10 



20 



30 



[0121] ^tc, m«^jastcoi>TiKwrso 

C0 12 21H2 9B, ^)«aaiiflE070-ft-h 
^-TtcDTSSo ft, 0 2 9<D!7n-^— Mc&l^ 
Ttt, Wfc«f5©*V«»)/V9TB«^l«¥S5 1 7 

[0 12 3] |W)0tC*5V^T, /V^Tm^fSS 1 
7«, S-f, *— /^-^y^lBBfc, O&^fr-ltS^ 40 

ff3 (S 2 9 0 1) o 
[0 12 4] /S— ^•y7 p j6HOfS5£tcOV''T 

x' =Ax + By + C 
y' =-Bx+Ay+D 

[0 12 5] 0HX.Iflg 1 9lC7jk?£5fc-£lcm.W?%W 
i300 4A < 64 0X4 80 F7 h<DW@& & 
t(639, 0) t (6 3 9, 4 7 9) tt±®3. (x, so 



y) IcRXLZmznrz (xl' p yl'), (x 

2' , y2' ) 7b\ &<D|Sft3 0 0 5<D*— r^-^vZf 

®.mt%.2> 0 ircfre, ^-^-77^13 0 0 2^ 

-0300 1^ 0^:€'&t5^?>7l'>i:LT*«)5>tX 

«H©f*3^&*5^i;i6i5Be«nfcm3t*io8iW3 o 

i>-oTm&3 0 0 3fT5o 

[0 12 6] #fc, 3»±fltOM40^ g«$BH*& 
0*^«glJ-TS tX h ^7AJBI^t5 ( S 2 9 0 
2) „ 

[0127] ^— 5 * -fmmn 3 o o 2 <om 

T\ M^r©ffl« (0!l*.fcf;fc«l<E>B«fc) o^tcjttLTt'i 

[0 12 8] 0 2 ORtf0 2 1 fc)ifg£>fc:Xl- 

K a 2, a 3©e*ISb 1, b 2, b 3%j£A63 0 ^ 
LT. b 1 tfb^Wit h 1 <fc<9*#< Cfc^fcJStoft 
«»»*-ffi«a±^A/T?^T) . b 2tf L^vMIt h 
2 £ *)>]^ < (X^TfelttTf t>*^i:JBton*WS«fl!) 
feO^-S»J:0^<) , b 3^LV^{fit h 3i0 

[0 12 9] ±SSO«t3^L,TX7 L y 7S 2 9 0 2T*£ 
*»H53b^J8UL (S 2 9 0 3) , C OfUSUttJIItc <fc o 

t, ->-Ai^xsas%-&*^v^fig5aa (S 2 9 0 
4) , xiti'-i>\sx%m%'£ts-e!!$.mmzi7^ (S2 

9 0 5), ffla*»T"r*. 

[0 13 0] &fC, StilBX^^yS 2 9 0 4<D->-AL 

[0 13 1] £1% 02 2%#MLT">-AUXj2ia^ 

[0 1 3 2] 0 2 2tc;in^:0^Lfc:tcD^-ro Biflt 
tf&mm 2 1 0 1, feli#2 1 0 2©f^ £H^2 1 
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0 1 CD 2 fg©* * * £-£fiStB<|!SHi§!t 2 1 0 3 £ LT5t« 
-f 3 D c©-&j£B<tiffi^2 l 0 3lc, £-f;fcB<f!2 l 0 

1 fcOfcStefcrt*©^:^ 0 0 1 

[0133] &tc, ^am»ffm*)<o9m (x. y) k 

x' =Ax + By + C 
y' =-B x+Ay + D 

KJ:tK *fjET3 (x' , y' ) **»5o ^LT£B 
»© (x' , y' ) (Dmmtt (x, y) fC3fc!--r£o 

cn*£j8H»««2 i o 3oaoo«iiTewu 

fir5o 

[0 13 4] S3 0«, 
^Ill07n-f t- h%it t©T'*5o ffi. 03 

y ^ vH«-&ia#a 5 1 7 ^©snawf^? 3 * a ic 

[0 13 5] S-f, ft 1 ©B& (0 2 2-e<DjEmm <D 
2ffioWM««AB*1HK2:LTVMil-r« (S 2 2 o 

1) o ?LT, Si OI«©5'f>3 0 0 1 iOTifllJ© 
«W*C©#j*M«IWfc*Mfc3e— (S 2 2 0 

2) o 

[0 13 6] ^JfC, ^r/&B<$!©?S9©1Ki|!c (x, y) IC 

x' =Ax + By + C 
y' =-B x +A y + D 

tCiD, ttfW* (x' . y' ) *3i!£>% (S 2 2 0 

3) o *LT, Cx' , y' ) lt^2CDmm (B2 2T? 
<D£BflO mc2&ZfrE*>fr* 1 7-*>y*L (S2 20 

4) , fcttftfc?, tMff)Xf'y7S 2 2 0 6^813*51 
«K ^nif, (x' , y' ) ©Bjg£r (x, y) tcrntf 

(S 2 2 0 5) o 
[0 13 7] £P>tC, £lhOXfy7S 2 2 0 3fr£>S 

2 2 0 5tZT-<Dmm*-£im@lCD&Q<Dffi.m,£TlCiiL 
TM9j5lfT9tefr£5frftftWrL (S 2 2 0 6) , £ 

[0 13 8] SfcfC, |WWf»7'S 2 9 0 5©'>-Al/ 

[0 13 9] 0 3 1 «\ '>-AL^X5QIi^tyj®#-&^ 
il©7D- — hfcj^-f t>©7?fc3o 03 1© 

viiMfS 5 l 7 #*©fflS»fts*fr3 <fc 3 K&o 

[0 14 0] *f, SuI5Xx-y^S 2 2 0 1 i:|5]«ltc, 
ft 1 ©Bill! (0 2 2 T©;fcB«0 © 2 ft©ffitt&-&j£B 
Itffi^fcLTBifSf 3> (S 3 2 0 1 ) o »K, ">-Ab 

xaas^tT 9 «sh 3003*5 fc£flii©«i«e£:^B« 
«««cJW«te3t:-r* (s 3 2 o 2) „ 

[0 14 1] ^LT, S/— hVT.Vm^Vio (S 3 2 0 
3) o *itt>%. 
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x' =Ax + By + C 
y' =-B x + Ay+D 

ip K p 2%*a6§ 0 fLT, 
£-j£Bfitp 3 = (1-a) *pl+a*p2 

a = ~>-a uxmrnftT'commcD x ^©{u»/->— a 

[0 14 2] XT7 7"S 3 2 0 3T">- AUXMS^ 
£2©B{*®39Qi6tt*^JftB{KRift 
10 icmfcf-L (S 3 2 0 4) , MHSrifeST-f 3o WU CtD 
■^OTJti, @30O7f77S 2 20 3^?)S 2 206 

$H#£fFfi£-f 3 C t 7?£ So 
[0 14 3] JW±KMLfcJ:3te*#BMfi©«!BteJ:n 

tfx /v^v^jaHQSfcte/SoT, ^j«f *b«**£* 

£{£© t, ©T&3 ©fr 2 5 A^f iJBU U £ WBf 

^tifc t %\z{ti/—K uxmrnant)^ c t «t o , 

20 v<>$IBHBtf*i*-e£*. 

[0 14 4] (m2confficDBM) #fc, *»^»cJ:* 
$6 2 ©*jfi©Jg«l*KWr 5o 

[0 14 5] WTte»2©*S6©*»4:LT, *^B« 
©i:^©o^€-a-fe-&©7-r>%. 

[0 1 4 6] £CD%$2CDnMCDBmTCDMl8.RUW} 
mi, 3Me«fc£l©*M©JBa8fcR«T**!K 01 9 
T'©3 0 0 1 ©o&^fr-£<D^-r>£:;)<i6£> t C 5fc 
30 W-tfgfcS. iotCLTa, Sl©*Sg©^ffitliS 

[0 14 7] HuKm 1 ©#lfiS©Jgffi7?«;/V^^j£# 
IS 5 l 7 ldit)^t>-&©{ft«*XA,*©5'f>fc: 

— ^•y7 p ©4'(D^t)f#§g£0-&t)-l*©ilil2 8 0 2 ©5 
5o 

40 [0 14 8] ^K**jfi©je»T»©»f^%BiWrS, 

[0 1 4 9] 03 3(c4M0^tt?®tt«n&flS#® 
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* NOTICES * 

Japan Patent Office is not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may 
not reflect the original precisely. 

2. **** shows the word which can not be translated. 

3. In the drawings, any words are not translated. 



CLAIMS 



[Claim(s)] 

[Claim l] The panorama image synthesizer unit characterized by to have a 
distinction means distinguish whether it is the image with which an image of 
said duplicate image field makes an alphabetic character a subject in the 
panorama image synthesizer unit which compounds two or more images 
which have an image field which overlapped mutually, and creates one 
synthetic image with a large field angle, and a synthetic means perform said 
image composition after performing a different image processing according 
to this distinction result. 

[Claim 2] A panorama image synthesizer unit characterized by having a 
synthetic means to compound two or more images which have an image field 
which overlapped mutually, to add shading-off processing to a boundary 
portion which connects said two or more images in a panorama image 
synthesizer unit which creates one synthetic image with a large field angle, 
and to perform said image composition. 

[Claim 3] While compounding two or more images which have an image field 
which overlapped mutually and creating one synthetic image with a large 
field angle When a binary printer printed by error diffusion method as said 
printer is specified in a panorama image synthesizer unit printed by printer 
which had this synthetic image specified A panorama image synthesizer unit 
characterized by having a synthetic means to perform said image 
composition bordering on Rhine where brightness of an image in said 
duplicate image field is the highest. 

[Claim 4] It is the panorama image synthesizer unit according to claim 3 
which has a distinction means distinguish whether it is the image with 
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which an image of said duplicate image field makes an alphabetic character 
a subject, and is characterized by for said synthetic means to perform said 
image composition after performing a different image processing according 
to a distinction result of said distinction means, when a binary printer 
printed by error diffusion method as said printer is not specified. 
[Claim 5] Said synthetic means is a panorama image synthesizer unit 
according to claim 1 or 4 characterized by performing said image composition 
without performing seamless processing to an image of said duplicate image 
field when said distinction means was the image which does not make an 
alphabetic character a subject and it distinguishes, and performing seamless 
processing, when it was the image which makes an alphabetic character a 
subject and distinguishes. 

[Claim 6] Said distinction means is a panorama image synthesizer unit 
according to claim 1, 4, or 5 characterized by an image of said duplicate 
image field distinguishing an alphabetic character based on luminance 
distribution of an image [ in / for whether it is the image made into a subject / 
said duplicate image field ]. 

[Claim 7] Said synthetic means is a panorama image synthesizer unit 
according to claim 1, 4, 5, or 6 characterized by not performing seamless 
processing to an image of said duplicate image field when said distinction 
means was the image which makes an alphabetic character a subject and it 
distinguishes, but performing said image composition bordering on Rhine 
where brightness of an image in said duplicate image field is the highest. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[The technical field to which invention belongs] This invention compounds 
two or more images which some images overlap, and relates to the panorama 
image synthesizer unit which creates one synthetic image. 
[0002] 

[Description of the Prior Art] Two or more images which some images 
overlap (duplication) are compounded on a computer, and, generally 
processing in which one synthetic image is created is called panorama 
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composition. This can be called processing from demand of wanting to photo 
a wide image and to make it the image of one sheet. Moreover, in an 
electronic camera, as demerit in comparison with a film-based camera or a 
scanner, although low [ of resolution ] (little of a pixel number) is pointed out 
therefore, it is important [ panorama composition ] for the image photoed 
with this electronic camera also as a means which photographs a high 
resolution image [ photograph / not only / a wide image ]. A manuscript, a 
magazine, etc. of a sheet are divided into plurality, and are photoed, and, 
specifically, power is demonstrated for it to divide scenery into plurality and 
be wide, and take [ acquire about the same high resolution data as a scanner, 
and ] a photograph to high resolution. 

[0003] Here, in panorama composition, the processing which blots out the 
knot is important and the grace of the done synthetic image is influenced 
greatly. Processing (it is henceforth called "seamless processing".) which 
blots out a knot as shown in drawing 38 as general technique is performed. 
That is, in the portion to overlap, according to the location of a pixel, a 
synthetic ratio is changed gradually, and it adds so that both images may 
become by a unit of 50% in the location of middle. When overlap is large, as 
shown in drawing 40 , it is predetermined width of face, and seamless 
processing is performed. 

[0004] This seamless processing is effective in especially natural drawings, 
such as scenery, and the high image of grace without a knot is obtained. 
[0005] 

[Problem(s) to be Solved by the Invention] However, there were the following 
problems in the above Prior arts. 

[0006] Although the synthetic location of the image overlapped using the 
method of extracting corresponding points, among two or more images to 
compound is decided in the case of panorama composition, an error is 
somewhat included in a synthetic location at this time. That is, the minimum 
extract precision at the time of the extract of corresponding points is 1 pixel, 
the precision not more than it cannot be guaranteed, but, therefore, a gap of 
less than 1 pixel produces it as a synthetic location of a pixel. 
[0007] Moreover, in the case of the image photoed with the electronic camera 
etc., distortion of some of images has produced the image of the portion 
located in the periphery of a lens, and a gap is included in it also in this 
reason. 
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[0008] On the other hand, in the image with alphabetic characters, such as a 
manuscript, the contrast of the white of paper and the black of an alphabetic 
character has clarified. If this manuscript is compounded and carried out 
and seamless processing is performed, each is transparent and is seen by the 
alphabetic character of the portion which performed seamless processing as 
shown in drawing 25 , and since the contrast of an alphabetic character is 
high, that gap will look notably. However, since it is more desirable to be low 
connected smoothly by contrast compared with the alphabetic character 
image in the case of natural drawing, seamless processing is effective. 
However, about a thing with alphabetic characters, such as a manuscript, 
the opposite effect is induced as mentioned above in many cases. 
[0009] Then, it is made in order that this invention may solve such a problem, 
and though it is a simple configuration, it aims at offering the panorama 
image synthesizer unit which can compound two or more images good. 
[0010] 

[Means for Solving the Problem] After performing an image processing 
which is different from a distinction means distinguish whether it is the 
image with which the image of said duplicate image field makes an 
alphabetic character a subject in the panorama image synthesizer unit which 
this invention compounds two or more images which have an image field 
which overlapped mutually, and creates one synthetic image with a large 
field angle, according to this distinction result in order to solve the 
above-mentioned technical problem, it is characterized by to have a synthetic 
means perform said image composition. 

[001 1] In addition, said synthetic means performs seamless processing to an 
image of said duplicate image field, when said distinction means was the 
image which does not make an alphabetic character a subject and it 
distinguishes, and it may be made to perform said image composition 
without performing seamless processing, when it was the image which 
makes an alphabetic character a subject and distinguishes. Moreover, you 
may make it an image of said duplicate image field distinguish said 
distinction means based on luminance distribution of an image [ in / for 
whether it is the image which makes an alphabetic character a subject / said 
duplicate image field ]. 

[0012] Moreover, this invention is characterized also by having a synthetic 
means to compound two or more images which have an image field which 
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overlapped mutually, to add shading-off processing to a boundary portion 
which connects said two or more images in a panorama image synthesizer 
unit which creates one synthetic image with a large field angle, and to 
perform said image composition. 

[0013] Furthermore, while this invention compounds two or more images 
which have an image field which overlapped mutually and creating one 
synthetic image with a large field angle When a binary printer printed by 
error diffusion method as said printer is specified in a panorama image 
synthesizer unit printed by printer which had this synthetic image specified 
It is characterized also by having a synthetic means to perform said image 
composition bordering on Rhine where brightness of an image in said 
duplicate image field is the highest. 
[0014] 

[Embodiment of the Invention] Hereafter, the gestalt of operation of this 
invention is explained to details with reference to a drawing. 
[0015] (Gestalt of the 1st operation) The gestalt of the 1st operation by this 
invention is explained first. 

[0016] Drawing 1 is the outline external view showing the example which 
applied the panorama image synthesizer unit by this invention to the 
personal computer. 

[0017] As shown in this drawing, this equipment has the main part 301 of a 
computer, with the display 302 which displays the various data containing 
the image concerning panorama composition respectively connected to this, 
the mouse 303, and the keyboard 305, possesses an electronic camera 307 
further and is constituted. 

[0018] Here, a mouse 303 is a typical pointing device and is equipped with 
the mouse button 304. 

[0019] Moreover, an electronic camera 307 has the internal memory which 
records the information about the image which self photoed, and is connected 
to the main part 301 of a computer by the general interface 306 in which an 
image transfer at high speeds, such as a bidirectional parallel interface and a 
SCSI interface, is still more possible. And an electronic camera 307 is set to 
panorama image photography mode when photoing the image applied to 
panorama composition apart from the time of the usual photography. 
[0020] Drawing 2 is the block diagram showing the configuration of this 
equipment containing software and hardware. 
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[0021] said drawing - being shown " as - a book - equipment hardware 
-- 509 - this " hardware - 509 -- a top - operating - an operating system 
(-- OS --) - 505 - this - OS - 505 - a top - operating - application software 
-- 504 - having - constituting - having - **** . In addition, although 
naturally contained as requirements for a configuration among the blocks 
which constitute hardware 509 and OS505, when explaining the example of 
a gestalt of operation of this invention, it is not illustrating about the block 
which is not needed directly. As an example of such a block that is not 
illustrated, there is a memory management system etc. in CPU, memory, 
and OS505 in hardware 509. 

[0022] Here, OS505 possesses the input device managerial system 506, the 
drawing managerial system 507, and the file system 508. 
[0023] The input device managerial system 506 has the function to enable it 
to receive a user's input, without application software 504 being conscious of 
hardware 509. 

[0024] The drawing managerial system 507 has the function in which 
application software 504 enables it to draw without being conscious of 
hardware 509. 

[0025] A file system 508 has the function the application software fair 504 
enables it to output and input a file, without being conscious of hardware 509. 
[0026] Moreover, hardware 509 possesses a keyboard interface 510, the 
mouse interface 512, the video interface 513, the disk IO interface 514, the 
hard disk (HD) 515 that stores a file and data physically, and the general 
interfaces 306, such as a bidirectional parallel interface or a SCSI interface. 
[0027] Here, the disk IO interface 514 is for reading and a file system 508 to 
write HD515. 

[0028] The video interface 513 is for the drawing managerial system 507 to 
draw on a display 302. 

[0029] A keyboard interface 510 is for the input device managerial system 
506 to receive the input of a keyboard 305. 

[0030] The mouse interface 512 is for the input device managerial system 
506 to receive the input from a mouse 303. 

[0031] An electronic camera 307 can be connected to the main part 301 of a 
computer, an electronic camera 307 can let the input device managerial 
system 506 pass by this, and a general interface 306 can exchange image 
data etc. now. 
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[0032] Moreover, application software 504 possesses the image data 
management system 501 with the data control means 502 and the data 
display means 503, and the panorama image composition means 517. 
[0033] Here, the data control means 502 is for managing image data by 
attribute information or the keyword by a user's input. 
[0034] The data display means 503 searches and expresses the image data 
managed as the attribute information or the keyword by a user's input. 
[0035] Moreover, the panorama image composition means 517 registers the 
image of the result of performing the image photoed in panoramic exposure 
mode through the image data management system 501, and having 
compounded panorama image composition processing for it further in 
reception and three kinds of synthetic modes (full auto composition, auto 
composition, and semi auto composition) mentioned later, to the image data 
management system 501. 

[0036] The function as a distinction means to distinguish whether it is the 
image with which the image of the image field where the panorama image 
composition means 517 furthermore overlapped makes an alphabetic 
character a subject is realized, and further, after performing a different 
image processing according to this distinction result, the function as a 
synthetic means to perform said image composition is realized. 
[0037] The panorama image composition means 517 as a synthetic means 
compounds like drawing 3 without performing seamless processing, when 
judged as an alphabetic character subject by the distinction result. That is, it 
has come to stick an image bordering on Rhine 2601 of the middle of an 
overlap location. Although some gap is produced in the stuck alphabetic 
character portion by this technique, since the amount of the part which has 
shifted becomes less, compared with the result in seamless processing as 
shown in drawing 40 of the conventional example, it serves as a synthetic 
high image of quality more. In addition, when not belonging to an alphabetic 
character subject, the usual seamless processing is performed. 
[0038] Although it distinguishes that it is that the image to compound 
belongs to an alphabetic character subject, the panorama image composition 
means 517 as a distinction means takes the histogram of the brightness of an 
image, and distinguishes distinction of being that this image to compound 
belongs to an alphabetic character subject. That is, if it seems that it judges 
according to luminance distribution as shown in drawing 4 and drawing 5 , 
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for example, luminance distribution is distributed over the whole like 
drawing 5 , it will be judged as natural drawing. Here, the range which takes 
the histogram of brightness is good in the range which an image overlaps, 
and since it is not necessary to cross to the whole image, improvement in the 
speed of processing can be attained. 

[0039] Next, the data structure of the information about the image which the 
electronic camera 307 photoed stored in the internal memory in an electronic 
camera 307, i.e., image data, and attribute information is explained using 
drawing 6 . 

[0040] Although a photograph is taken with the gestalt of this operation after 
a user sets an electronic camera 307 to "panorama image photography mode" 
in case an image is photoed with an electronic camera 307 as mentioned 
above, the identifier which shows an one-set panorama image is 
automatically recorded by making it this photography mode into the 
attribute information on the photoed image which is recorded on an internal 
memory. 

[0041] As shown in drawing 6 , the image managed table 81 is placed into an 
internal memory, it has the image data storage field 82 and the attribute 
information storing field 83, and attribute information is referred to with the 
photoed image data which corresponds for every image. 
[0042] here - the image data storage field 82 - image data 82a and 82b - 
format data (NEITIBU data) original with an electronic camera 307 - or it is 
stored with either of the general-purpose format data, such as JPEG, and a 
user can choose these either according to a photography condition, and can 
store now. In addition, NEITIBU data is data obtained by A/D [ the output 
from CCD ] only, and although the time amount which record takes is short, 
it has come to be able to enlarge data size generally. Moreover, JPEG data 
can make data size small, although the time amount which record takes 
unlike NEITIBU data is taken. 

[0043] File names 84a and 84b, file types 85a and 85b, the photography time 
86a and 86b, and the photography modes 87a and 87b are recorded on the 
attribute information storing field 83 as attribute information. 
[0044] Here, file names 84a and 84b are unique file names which an 
electronic camera 307 attaches automatically. 

[0045] It is shown whether file types 85a and 85b are other general-purpose 
formats which it being a NEITIBU data format or being a JPEG format or an 



8/32 



Japanese Publication number : 09-093431 



electronic camera 307 supports. 

[0046] The time and time amount at the time of the shutter release by which 
the photography time 86a and 86b was clocked by the calender and timer of 
the electronic camera 307 interior being pushed are recorded. 
[0047] The photography modes 87a and 87b show the photography mode 
chosen among some kinds of photography modes which an electronic camera 
307 has at the time of photography, and when this is in "panoramic exposure 
mode", Identifiers 88a and 88b are added further. 

[0048] The number-of- sheets information 90a and 90b which shows the how 
many sheets in the modes 89a and ID 89b which are the unique numbers set 
when these identifiers 88a and 88b are set to panoramic exposure mode, and 
the mode of those they are is stored. Therefore, it will be said that two or 
more images which have the same modes 89a and ID 89b in panoramic 
exposure mode are one set. In addition, in the example of drawing 6 , since 
scenery is photoed as an image of two right and left, it is the ID mode in 
which Modes 89a and ID 89b are the same. 

[0049] With this equipment, although image data and attribute information 
are stored in an electronic camera 307 as mentioned above, further, an 
electronic camera 307 is connected to the main part 301 of a computer, and 
the image + attribute information in an internal memory is copied to HD515. 
[0050] Next, a means to copy the data recorded in the electronic camera as 
mentioned above to a computer is described. 

[0051] Drawing 7 shows the screen when copying the data in an electronic 
camera 307 to HD515. 

[0052] This screen is displayed on a display 302 by the image data 
management system 501, and a window 91 and a window 92 are displayed. 
[0053] The data of the internal memory in the electronic camera 307 called a 
camera catalog is displayed on a window 91, and the contraction image 
(thumbnail image) 94 and the attribute information- display field 95 of image 
data are displayed on it. Moreover, the frame 93 which indicates that the 
window 91 was chosen when an image was chosen by the user is displayed. 
[0054] The file name in attribute information, a file type, etc. are displayed 
on the attribute information- display field 95. In addition, it can be changed 
now by user assignment how far it displays among this attribute information. 
[0055] The data called the user catalog which are some image databases of 
the user who exists in HD515 is displayed on a window 92. Moreover, a copy 
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will be performed, if a user chooses an image from windows 91 and, as for a 
window 92, operates Drag&Drop. 

[0056] In addition, it changes [ (the data in an electronic camera 307 is 
eliminated), and ] by assignment of a user whether it is a copy and whether 
they are (data remains in an electronic camera 307), and migration to both at 
this time. Moreover, in the midst of this copy actuation, NEITIBU data is 
changed into a predetermined general-purpose format with the image data 
management system 501, and if there is an image further photoed by the 
panorama image management tool 517 in panoramic exposure mode, those 
composition will be performed if needed. 

[0057] Next, it describes what kind of data structure the data copied as 
mentioned above has in the user catalog. 

[0058] Drawing 8 shows the data structure on the user catalog displayed on a 
window 92. 

[0059] A user catalog attaches the ID number of a proper about the image 
data stored in the interior with the image data management system 501, and 
is managed. That is, correspondence with the image data and attribute 
information which are linked to the ID number is taken, and, thereby, 
probability of the base of management is carried out. 

[0060] Moreover, for every user catalog of this piece, although, as for a user 
catalog, a user can have the number of arbitration, as shown in drawing 8 , 
the catalog table 1100 is prepared. 

[0061] Here, in this catalog table 1100, the data ID 1101 of the image 
belonging to this catalog and the group ID 1102 of the group who belongs are 
held. 

[0062] Moreover, group ID 1102 links with the group attribute table 1103. 
[0063] This group attribute table 1103 is fundamentally the same as the 
catalog table 1100, and has the group ID belonging to the data ID 1105 or 
this group of the image belonging to this group. In addition, the places where 
the difference from the catalog table 1100 has the group attribute data 1104 
in the head differ. 

[0064] As for this group attribute data 1104, a group name 1106, the date 
and time of creation 1107, and a group type 1110 are stored. 
[0065] When the identifier of the arbitration which the user attached 
attaches a group name 1106 and a group is made as a set of a panorama 
image, this group name is attached with a "panorama image" by the default. 
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[0066] As for the date and time of creation 1107, time when a group is 
created is stored. 

[0067] When a user creates a group type 1110 and a group is made as "user 
creation" and a set of a panorama image, the information of a "panoramic 
exposure" enters. However, it links with an identifier further at the time of a 
panorama image, and mode ID89a is dedicated. 

[0068] Moreover, actual image data and attribute information are stored on a 
user catalog with the same structure as the image managed table 81 of 
above-mentioned drawing 6 . That is, these are referred to from the data 
control table 1108, further, it is linked to data ID 1109 within the data 
control table 1108, and correspondence with each image data and attribute 
information is taken. 

[0069] As mentioned above, in this equipment, it has the function in which a 
user does [ the image data in a user catalog ] a category division for two or 
more images as one group. Namely, thereby, the inside of one user catalog is 
hierarchized and data can be managed now. 

[0070] Next, three kinds of synthetic modes concerning the panorama image 
composition processing by the panorama image composition means 517 are 
explained. 

[0071] With this equipment, an electronic camera 307 is connected to the 
main part 301 of a computer, and in case the image + attribute information 
in an internal memory is copied to HD515, this attribute information is 
checked by the image data management system 501. Here, from that to 
which the identifier in panorama image photography mode exists in the 
attribute information on a user catalog, an one-set image is extracted 
automatically, next processing of panorama image creation is started. 
However, since it has two or more synthetic modes of image composition 
processing in this invention, next synthetic mode is chosen. 
[0072] When the number of images is two, this synthetic mode with the full 
auto composition which compounds with a perfect automatic at the time of 
three or more sheets When corresponding points are not enough called for in 
the check phase of the auto composition which has a user specify only the 
relative position of the four directions of an image and these full auto 
composition, or auto composition, Or when a user wants to save the time 
amount which detection of corresponding points takes, and to perform 
synthetic processing by shorter time amount, it consists of semi auto 
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composition to which composition is carried out because a user specifies a 
great portion of overlap location. 

[0073] Here, one set of a panorama image is extracted and full auto 
composition is the mode chosen when it is two sheets. In this full auto 
composition, since four kinds of four directions can be considered as a 
synthetic location of the image of two sheets as shown in drawing 9 thru/or 
drawing 12 , processing which asks for the corresponding points of the 
portion overlapped in [ these ] four is performed, and the place where a lot of 
corresponding points which are in agreement more than predetermined level 
were able to be found is compounded as a right composition location. In 
addition, in this synthetic mode, as actuation of a user, it is only the 
actuation which copies an image into a computer from an electronic camera 
307, and the panorama composition means 517 will do the rest automatically. 
Moreover, since two-sheet composition is usually considered to be most 
except a special use, this synthetic processing is performed in most cases. 
Furthermore, in the gestalt of this operation, since it can say that certainty 
is low when the number of the corresponding points which are in agreement 
more than predetermined level in any [ four ] case when performing 
processing which asks for corresponding points is below a predetermined 
amount, at this time, full auto composition processing is closed and it shifts 
to semi auto composition processing. 

[0074] Auto composition is the mode which one set of a panorama image is 
extracted, and is chosen when it is three or more sheets. And in this auto 
composition, it displays on the window as a user interface as shows an 
one-set image to drawing 13 and 14. All the images that belong to the group 
of a panorama image on a screen are first resized and displayed on the 
magnitude included in a window, a user looks at this, it Drag(s) in right 
sequence, only the physical relationship of the upper and lower sides and 
right and left is directed in this window, and it rearranges into it. For 
example, in the example of drawing 14 , since the image located under a 
window 1401 becomes the rightmost in fact, if it Drag(s) on left-hand side, 
the panorama image composition means 517 will detect that it is the 
panorama horizontally located in a line three from the location. And as it has 
gone into the window, it is resized again and displayed like a window 1402. 
That is, corresponding-points extract processing in which it asks for the 
corresponding points of each image is directed based on a user. In addition, 
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in the gestalt of this operation, when the number of the corresponding points 
which are in agreement more than predetermined level with the panorama 
image composition means 517 in the case of the corresponding-points extract 
processing by auto composition is more than a predetermined amount, it 
compounds as a right composition location, otherwise, since certainty is low, 
auto composition processing is closed, and it shifts to semi auto composition 
processing. 

[0075] Semi auto composition is the mode chosen when a user wants to save 
the time amount which a corresponding-points extract takes, and to obtain a 
synthetic result early more in full auto composition or auto composition 
processing when the certainty of a corresponding-points extract is low or. 
And in this semi auto composition, a user drag(s) the image displayed on the 
window as a user interface as shown in drawing 15 and drawing 16 , and 
specifies the location of the great portion of overlap. That is, in this semi auto 
composition, based on [ time / of auto composition processing ] the positional 
information specified by the user, corresponding-points extract processing in 
which it asks for the corresponding points of each image performs 
corresponding-points extract processing in the much narrow range, asks for 
a location most in agreement, and performs synthetic processing further 
from this result. For example, in the example of drawing 16 , as shown in a 
window 1801, all the images that belong to the group of a panorama image 
on a screen are first resized and displayed on the magnitude included in a 
window 1801, and a user looks at this, and as shown in a window 1802, a 
great portion of overlap location is doubled, and it piles up. Since the place 
piled up mutually displays by carrying out an AND operation for every bit 
per pixel, both images are transparent and the overlapping portion is visible. 
And it is resized again and it is displayed that it has gone into the window. 
In addition, the actuation by the semi auto composition on this window is 
fundamentally the same as actuation by auto composition, and there are few 
burdens of a user. A difference is only the point (semi auto composition 
processing) of overlapping that (auto composition processing) which 
rearranges and displays the location which Drag(ed) and detached the image 
with pointing devices, such as a mouse, only using the information on 
physical relationship, using the information on a location as it is, and 
displaying. Moreover, since it is transparent and visible by the previous AND 
operation also in Drag, a great portion of location can be put together easily. 
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[0076] In addition, after corresponding-points extract processing finishes in 
the case of which [ of the above-mentioned synthetic mode ] and the overlap 
range of an image is called for, the way of seamless processing is changed 
and synthetic processing is performed so that it may distinguish whether it 
is an alphabetic character subject's manuscript image, and whether it is the 
usual natural drawing, next may mention later based on this distinction 
result. Moreover, although a user performs actuation for a 
corresponding-points extract as mentioned above in auto composition and 
semi auto composition, since it is only actuation of both drag(ing) an image 
and is common actuation that this is the simplest and, a user's burden is 
small. Moreover, since an image is drag(ed) and it only doubles with a great 
portion of location at the time of semi auto composition, it is much easy and 
easier than actuation of specifying the point of the conventional example 
clearly. 

[0077] Next, actuation of the gestalt of this operation is explained according 
to the flow chart of drawing 23 thru/or drawing 31 , referring to drawing 17 
thru/or drawing 22 . 

[0078] First, the actuation at the time of copying the image data in an 
electronic camera into a computer is explained. 

[0079] Drawing 23 is the flow chart of the processing performed in case the 
image data in an electronic camera is copied into a computer. In addition, in 
the flow chart of drawing 23 , especially, as long as there is no notice, the 
image data management system 501 performs the processing actuation. 
[0080] In this drawing, in order that a copy may process to all required 
images first, it judges whether processing has finished to all images (S1000). 
If processing has finished, processing is advanced to the below-mentioned 
step S1009, and if it has not finished yet, processing will be advanced to the 
following step S1001. 

[0081] First, the attribute information which accompanied the image data of 
a piece and it first is acquired in copy actuation (S1001). From the file types 
85a and 85b within attribute information, this image data judges whether it 
is NEITIBU data (S1002). If it is not NEITIBU data, processing will be 
advanced to the below-mentioned step S1004. Moreover, NEITIBU data will 
be changed into the general-purpose formats (JPEG, TIFF, etc.) decided as a 
default if it is NEITIBU data (S1003). And if conversion finishes, file types 
85a and 85b will also be updated. 



14/32 



Japanese Publication number : 09-093431 



[0082] Next, it confirms whether to be the image which investigated the 
photography modes 87a and 87b, and was photoed in panoramic exposure 
mode (S1004). When it is not a panorama image, it registers as usual image 
data (S1008). The data ID of a proper is specifically attached and registered 
into the data control table 1108 in drawing 8 , and the data ID is registered 
into the catalog table 1100. 

[0083] When it is a panorama image, it confirms whether to be finishing [ the 
group for these panorama images / creation ] already (S1005). This follows 
the catalog table of drawing 8 , and when mode ID89a of group ID looks at 
whether it is the same as mode ID89a of an image, it is performed. 
[0084] A group is created when there is no corresponding group (S1006). This 
newly registers group ID 1102 into the catalog table 1100, and creates a 
group name 1106, the date and time of creation 1107, and a group type 1110. 
It is recorded on a group type 1110 as a "panoramic exposure", and mode 
ID89a in the attribute information on an image is dedicated. 
[0085] And the data ID of a proper is attached to this panorama image data, 
and it registers with the managed table 1108, and registers with data ID 
1105 (S1007). 

[0086] It carries out from the above step S1000 to all the images that copy a 
series of processings of S1008. If processing finishes to all images, although 
copied until now, if it confirms whether the group of a panorama image was 
made (S1009) and it is made, the panorama image composition means 517 
will perform in inside panorama image composition processing which 
mentions later using the image in a group (S1010). If there is nothing, 
processing will be ended now. 

[0087] Next, explanation about the panorama image composition processing 
in said step S1010 is given. 

[0088] Drawing 24 shows the flow chart of panorama image composition 
processing. In addition, in the flow chart of drawing 24 , especially, as long as 
there is no notice, the panorama image composition means 517 performs the 
processing actuation. 

[0089] In this drawing, the panorama image composition means 517 
confirms first whether there are more images in a group than two sheets and 
two sheets (S1200). At the time of two sheets, the full auto composition 
processing mentioned later is started (S1202). When [ than two sheets ] more, 
the auto composition processing mentioned later is started (S1201). After 
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each processing at these steps S1201 and S1202 finishes, the result of that 
synthetic processing checks a success or failure (S1203, S1204). Decision of 
this success or failure is judged by the ability of the point with which images 
correspond to have been found enough. Therefore, in the synthetic whole 
processing, since a judgment is made in an early phase, failure of the time 
amount for which it waits until a user obtains the result is also short, and a 
result can be managed. And if it is a success, processing is termination, and 
processing is ended after performing semi auto composition processing 
mentioned later (S1205), if it is failure. 

[0090] Next, explanation about the full auto composition processing in said 
step S1201 is given. 

[0091] Drawing 25 shows the flow chart of auto composition processing. In 
addition, in the flow chart of drawing 25 , especially, as long as there is no 
notice, the panorama image composition means 517 performs the processing 
actuation. 

[0092] In this drawing, the panorama image composition means 517 acquires 
first the information on the physical relationship which the user rearranged 
(S1301). Next, the range searched although corresponding points in 
agreement are found, i.e., a matching range, is set up (S1302). In addition, as 
the Ruhr when taking a photograph as a panorama image, a minimum of 
10%, if making it overlapping a maximum of 50% and the gap of a direction 
which carries out a right angle to it are decided to be 5% or less, respectively, 
as shown in drawing 17 , the surely overlapped range will turn into a range 
shown in the slash area 1504 of the left image 1501 of drawing 17 . Moreover, 
the range which may be overlapped turns into a range shown in the slash 
area 1505 of the right image 1502. Now, when the point shown in the point 
1503 all over the area of the slash area 1504 follows the above-mentioned 
Ruhr, it will have a point [ be / it / under / area / which is shown in the search 
range 1506 in the area of the slash area 1505 / correspondence ]. It will be 
found in the corresponding-points extract processing mentioned later 
whether it matches to this area. 

[0093] A set of the parameter which will be used for the entry which more 
than searchs at said step S1302 if it returns to a flow chart performs 
processing which extracts corresponding points next (S1303). In addition, 
about the details of this corresponding-points extract processing, it mentions 
later. Since the number of the corresponding points which were able to be 
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found was not able to judge whether it was beyond a predetermined value (N 
individual) (S1304) and corresponding points were not able to be enough 
found automatically at the time of below a predetermined value after 
corresponding-points extract processing finished, it progresses to semi auto 
composition processing. When [ than a predetermined value ] more, it 
progresses to synthetic parameter setup processing (S1305). In this synthetic 
parameter setup processing, it asks for the parameter of the migration used 
in the case of composition, expansion (contraction), and rotation from the 
coordinate of previous corresponding points. In addition, these details are 
mentioned later. Finally, image composition processing is performed based 
on these parameters (S1306). These details are also mentioned later. 
[0094] Next, explanation about the full auto composition processing in said 
step S1202 is given. 

[0095] Drawing 26 shows the flow chart of full auto composition processing. 
In addition, in the flow chart of drawing 26 , especially, as long as there is no 
notice, the panorama image composition means 517 performs the processing 
actuation. 

[0096] In this drawing, although the panorama image composition means 
517 performs a matching entry first (S1601), as for this, said step S1302 and 
processing are the same. 

[0097] Next, corresponding-points extract processing is performed 4 times. 
Since number of sheets is limited to two sheets in full auto composition 
processing, an image 1 and the image 2 of the physical relationship 
considered are four kinds of right and left and right and left the upper and 
lower sides and on the bottom. Then, corresponding-points extract processing 
is carried out about these four cases, and the numbers currently extracted as 
corresponding points, respectively and such average coincidence level are 
held. These processings are processings from step S1602 to S1609. 
[0098] And it is confirmed whether, in [ four ] the upper case, there are some 
which have extracted the corresponding points beyond a predetermined 
value (N individual) (S1610). If there is no one, it will progress to semi auto 
composition processing. If it is, suppose in it that it is the best thing of 
average coincidence level true physical relationship (S1611). In addition, 
although it should become only one of four by the usual image when 
corresponding points are beyond predetermined values, a character which 
was alike, for example when a picture of a manuscript etc. was divided and 
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taken is located in a line, and beyond a predetermined value can be extracted 
as corresponding points also in the time of the physical relationship which is 
not right. Then, what fits most (what has the highest average coincidence 
level) is chosen at this step S1611. 

[0099] Although it progresses to the next synthetic parameter processing 
(S1612) and image composition processing (S1613) after processing of step 
S1611 finishes, this is the same processing as said steps S1305 and S1306, 
and is mentioned later for details. 

[0100] Next, explanation about the semi auto composition processing in said 
step S1205 is given. Processing here is almost the same as that of the time of 
auto composition processing. 

[0101] Drawing 27 shows the flow chart of semi auto composition processing. 
In addition, in the flow chart of drawing 27 , especially, as long as there is no 
notice, the panorama image composition means 517 performs the processing 
actuation. 

[0102] In this drawing, the panorama image composition means 517 acquires 
first the overlap positional information which the user of an image doubled 
(S1701). And although a matching range is set up (S1702), a range here 
turns into a predetermined range (error-range + margin of the location which 
the user assumed doubled). Therefore, it becomes a range much narrower 
than the range at the time of auto composition processing, and compaction of 
computation time and improvement in precision are achieved. 
[0103] Although it progresses to the next corresponding-points extract 
processing (S1703), synthetic parameter setup processing (S1704), and 
image composition processing (S1705) after processing of step S1702 finishes, 
each of these is the same as that of the time of auto composition processing. 
[0104] Next, explanation about corresponding-points extract processing is 
given. 

[0105] First, the outline of corresponding-points extract processing is 
described with reference to drawing 18 . 

[0106] Drawing 18 shows the left in the case of a corresponding-points 
extract, and the example at the time of two right images here. When two 
number of sheets of an image is large, since what is necessary is just to 
repeat composition of two sheets several times, as processing, it is 
fundamentally the same. 

[0107] First, the range 2005 which follows the Ruhr at the time of 
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photography and sets up a template is set as a 90% long and 10% wide range. 
Moreover, the range to search is the range of a possibility that the point of 
corresponding exists, and is set as 100% long and the 50% wide range 2006. 
From the area of the template setting range 2005 in an image, an edge looks 
for a strong point beyond a predetermined value, and starts length and a n 
pixels wide rectangle as a template image 2003 centering on there. This 
template image 2003 is placed on the search range 2004, and that difference 
is taken per pixel. It shifts 1 pixel of the search range 2004 tops at a time, 
and asks for the place where this sum total serves as min. If the minimum 
value of the result of having searched all search range tops is below a 
predetermined value, it will hold the points (x y), and (x\ yO of those as a pair 
of corresponding points. 

[0108] Although the above processing serves as an outline of 
corresponding-points extract processing, this is explained once again along 
with the flow chart of drawing 28 . 

[0109] Drawing 28 shows the flow chart of corresponding-points extract 
processing. In addition, in the flow chart of drawing 28 , especially, as long as 
there is no notice, the panorama image composition means 517 performs the 
processing actuation. 

[0110] In this drawing, the panorama image composition means 517 creates 
an edge extract image first (S1901). And an edge looks for the powerful point 
beyond a predetermined value from the range 2005 which sets up the 
template in this edge extract image (S1902). And if there is the point, an 
image will be cut down with the rectangle of every **n pixels of every 
direction from the point, and it will consider as a template image (S1903). 
[0111] Next, the search range in a right image is set up from the location of 
the point (S1904). And the difference of a pixel value takes an absolute value 
for the image and template image in a search range per superposition and 
pixel, and the sum total is searched for (S1905). 

[0112] Moreover, it confirms whether to be the minimum value till then 
(S1906), and if the total value of this difference is so, it will hold the 
coordinate and the minimum value of the point in that search range (S1907). 
A repeat and a point (it has the minimum difference) most in agreement are 
found out for the above in all search ranges. 

[0113] And it checks (S1908), and the minimum value calculated as a result 
is a sufficiently small value, or (are they clear corresponding points?) it 
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judges whether all search ranges were searched as compared with the 
predetermined value L (S1909). When smaller than the predetermined value 
L, the coordinate (x y) of the point which cut down the template image on the 
list of corresponding points, the coordinate (x f , yO, and the value of the 
minimum value of the point with which the minimum value was calculated 
are registered (S1910). 

[0114] If the above is performed to all template setting ranges (S1911) and it 
ends, the average value will be calculated from all the minimum values 
under list of corresponding points, and this is held as a coincidence level 
value (S1912). Corresponding-points extract processing is ended above. 
[0115] Next, synthetic parameter setup processing is explained. When an 
image is made into two sheets, (also in composition of two or more sheets, 
your thinking by two sheets first of all, since it is the repeat of composition of 
two sheets), and its gap can be expressed with advancing side by side of x 
and the direction of y, rotation, and the difference of a dilation ratio. 
Therefore, a point (x y), and corresponding (corresponding x\ y f ) can be 
expressed as follows. 
[0116] 

[Equation l] 



Here, in theta, the angle of rotation of the circumference of the Zaxis, and 
deltax and delta y show advancing side by side, and m shows a scale factor. 
Therefore, this coordinate transformation can be expressed by asking for 
Parameters A, B, C, and D. In previous corresponding-points extract 
processing, corresponding points (x y) and two or more groups of (x f , yO were 
acquired. This is asked for Parameters A, B, C, and D using a least square 
method. 

[0117] That is, it is the conditions of epsilon=sigma [{(Ax+By+C) 
-x'}2+{(-Bx+Ay+D) -y'}2] ->min, and is [0118]. 
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It asks for the ******** parameters A, B, C, and D. 

[0119] It is here and is PI =sigmax2+sigmay2P2 =sigmaxP3 =sigmayP4 
=sigmaxx , +sigmayyT5 =sigmaxy'-sigmax ! yP6 =sigmaxT7 =sigmay f P8 =n 
(the number of corresponding points). 

If it carries out, Parameters A, B, C, and D can be expressed as follows. 
[0120] 

[Equation 3] 



Parameters A, B, C, and D are computed by calculating P8 from this 
parameter PI, and substituting for a top formula. 
[0121] Next, image composition processing is explained. 

[0122] Drawing 29 shows the flow chart of the synthetic whole processing. In 
addition, in the flow chart of drawing 29 , especially, as long as there is no 
notice, the panorama image composition means 517 performs the processing 
actuation. 

[0123] In this drawing, the panorama image composition means 517 

performs the entry at the time of performing an overlap range, setup of 

Rhine to connect, and seamless processing first (S2901). 

[0124] About a setup of this overlap range, the formula and 

x -Ax+By+Cy -Bx+Ay+D which were calculated in the top are used. 

[0125] For example, if the image 3004 located in the left considers as the 

image of 640x480 dots as shown in drawing 19 , (x2*, y2') which substituted a 
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coordinate (639 0) and (639,479) for the top formula (x y), and were obtained 
(xl\ yl ? ) will become the overlap range of the right image 3005. From here, 
the overlap range 3002 can be found as a coordinate location of a left image, 
and it asks as Rhine which Rhine 3001 of the middle of this range connects. 
The range at the time of performing seamless processing is set up in quest of 
the field 3003 of the predetermined width of face set up beforehand from an 
overlap midrange. At this time, when the width of face of an overlap range is 
narrower than predetermined width of face, it has the width of face of an 
overlap range, and considers as a field 3003. 

[0126] Next, histogram processing which distinguishes whether it is an 
alphabetic character subject's image and whether it is a natural image is 
performed (S2902). 

[0127] First, the histogram of brightness is created about the pixel of overlap 
within the limits 3002. Since the same result only comes out at this time 
even if it performs to both images, it is easy to come out only to image (for 
example, left-hand side image) of one of the two. Therefore, a histogram 
setting range is narrow, and since it is only image of one of the two, the time 
amount which histogram processing takes is very short, and ends. 
[0128] Here, drawing which distinguishes whether it is the image of the 
alphabetic character subject who used the histogram of brightness for 
drawing 20 and drawing 21 , and whether it is a natural image is shown. As 
shown in these drawings, the shaft of the brightness of a histogram is 
roughly trichotomized and it asks for the total frequencies bl, b2, and b3 of 
each range al, a2, and a3. And bl is larger than a threshold thl (the portion 
considered to be an alphabetic character included more than the constant 
rate), and b2 is smaller than a threshold th2 (brightness considered not to be 
an alphabetic character or paper, either has few things than a constant rate), 
and when larger (the portion considered to be paper is included more than 
the constant rate) than a threshold th3, it is judged as an alphabetic 
character image and judged [ b3 carries out, and ] as a natural image except 
it. 

[0129] After finishing histogram processing at step S2902 as mentioned 
above next, this distinguishes whether it is an alphabetic character image 
(S2903), synthetic processing (S2904) which does not include seamless 
processing by this distinction result, or synthetic processing including 
seamless processing is performed (S2905), and processing is ended. 
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[0130] Next, the image composition processing which does not include 
seamless processing of said step S2904 is explained. 

[0131] First, the outline of the image composition processing which does not 
include seamless processing with reference to drawing 22 is described. 
[0132] What illustrated this to drawing 22 is shown. When images are the 
left image 2101 and the right image 2102, the image 2101 twice the 
magnitude of the left is secured as a synthetic image field 2103. The field to 
Rhine 3001 which connects the left image 2101 first is copied to this 
synthetic image field 2103 as it is. 

[0133] next - the remaining fields (x y) of a synthetic image - formula 
x^Ax+By+Cy-Bx+Ay+D - corresponding (x\ yO " it asks. And the pixel of 
(x\ yO of a right image is copied to (x, y). This is performed to all the 
remaining fields of the synthetic image field 2103. 

[0134] Drawing 30 shows the flow chart of the image composition processing 
which does not include seamless processing. In addition, in the flow chart of 
drawing 30 , especially, as long as there is no notice, the panorama image 
composition means 517 performs the processing actuation. 
[0135] First, a twice as many field as the 1st image (left image in drawing 
22 ) is secured as a synthetic image field (S2201). And the field on the left of 
Rhine 3001 of the 1st image is simply copied to this synthetic image field 
(S2202). 

[0136] next - the remaining fields (x y) of a synthetic image formula 
x-Ax+By+Cy-Bx+Ay+D - corresponding (x', yO - it asks (S2203). And it 
confirms whether (x ! , y f ) are in the 2nd image (right image in drawing 22 ) 
(S2204), and if there is nothing, processing is advanced to the 
below-mentioned step S2206, and if it is, the pixel of (x\ y') will be copied to 
(x, y) (S2205). 

[0137] Furthermore, it judges whether processing from the above step S2203 
to S2205 was repeatedly performed to all the remaining fields of a synthetic 
image (S2206), and if it goes to all fields, processing will be ended now. 
[0138] Next, the image composition processing including seamless processing 
of said step S2905 is explained. 

[0139] Drawing 31 shows the flow chart of the image composition processing 
including seamless processing. In addition, in the flow chart of drawing 31 , 
especially, as long as there is no notice, the panorama image composition 
means 517 performs the processing actuation. 
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[0140] First, a twice as many field as the 1st image (left image in drawing 
22 ) is secured as a synthetic image field like said step S2201 (S3201). Next, 
the field on the left of the range 3003 which performs seamless processing is 
simply copied to a synthetic image field (S3202). 

[0141] And seamless processing is performed (S3203). That is, first, a 
corresponding coordinate is searched for by formula 

x , =Ax+By+Cy , =-Bx+Ay+D, and it asks for the pixels pi and p2 to which both 
images correspond. And synthetic pixel p3=(l-a) *pl+a*p2a = the width of 
face of the location / the seamless range of the x directions of a pixel performs 
seamless processing within seamless limits. 

[0142] After finishing seamless processing at step S3203 next, the remaining 
fields of the 2nd image are copied to a synthetic image field (S3204), and 
processing is ended. In addition, this way is the same as that of processing 
from step S2203 of drawing 30 to S2206. Above, a final panorama 
composition image can be created. 

[0143] As explained above, the problem of the double counterpart in an 
alphabetic character can be solved by not performing seamless processing, 
when according to the gestalt of this operation the image to compound 
distinguishes whether it is belonging to an alphabetic character subject and 
is judged to be an alphabetic character subject in advance of panorama 
composition processing, and the problem in a panorama composition image 
that a knot will be conspicuous can be solved. 

[0144] (Gestalt of the 2nd operation) Next, the gestalt of the 2nd operation by 
this invention is explained. 

[0145] As a gestalt of the 2nd operation, how to set Rhine connected at the 
time of an alphabetic character image as Rhine with least (for brightness to 
be high) focus in an overlap range is explained below. 

[0146] In addition, the configuration in the gestalt of this 2nd operation and 
actuation are the same as that of the gestalt of the 1st operation 
fundamentally, and only the places which ask for Rhine which 3001 in 
drawing 19 connects differ. Therefore, here explains only a different portion 
from the gestalt of the 1st operation. 

[0147] With the gestalt of said 1st operation, with the panorama composition 
means 517, although the location of lamination was made into Rhine of 
middle, as shown in drawing 32 , Rhine 2801 with least focus is looked for 
among the width of face 2802 of lamination in overlap which can be taken, 
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and lamination is performed by the gestalt of this operation, there. Thereby, 
synthetic processing in which a knot is not further conspicuous can be 
performed. 

[0148] Next, actuation with the gestalt of this operation is explained. 
[0149] The flow chart of processing of the characteristic portion in the gestalt 
of this operation is shown in drawing 33 . In addition, this flow chart 
explains as a case of composition on either side. However, in the upper and 
lower sides, you may think the same way. 

[0150] In this drawing, the panorama image composition means 517 
initializes the variable max holding maximum to 0 first (S3301). And all the 
pixels belonging to vertical Rhine are acquired from the leftmost side of the 
overlap range shown by 3002 of drawing 19 (S3302). 

[0151] Next, it substitutes for Variable Sum in quest of total of these pixel 
values (S3303). in addition - since an image is an alphabetic character 
image here paper - white - or other bright colors - it is an alphabetic 
character - black - or it can be said that they are other dark colors. Then, it 
asks for total of a pixel for every Rhine, and it can be said that it is rare for 
Rhine of the largest value in it to divide an alphabetic character most. 
Moreover, although the filter of 3x3 may be let pass, calculated and 
determined in order to investigate the number of an alphabetic character 
edge for every Rhine for example, the simple addition using the feature of a 
previous alphabetic character image of the processing time may be shorter. 
[0152] And it judges whether Variable Sum is larger than Variable max 
(S3304), and if Variable Sum is larger than Variable max, it will be said that 
there are few portions of an alphabetic character, and Variable Sum will be 
substituted for Variable max and will be held (S3305). Moreover, the location 
pos of Rhine is also held. 

[0153] When the above is performed to the rightmost side of an overlap range 
one by one, the location pos currently held will call it Rhine with few portions 
of an alphabetic character. Rhine of this location is set up as Rhine 3001 in 
drawing 19 , and subsequent processings are continued. 

[0154] According to the gestalt of the 2nd operation, Rhine with least focus is 
looked for among the above width of face of the lamination which can be 
taken in overlap, and lamination is performed there. Thereby, synthetic 
processing in which a knot is not further conspicuous can be performed. 
[0155] (Gestalt of the 3rd operation) Next, the gestalt of the 3rd operation by 
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this invention is explained. 

[0156] As another means which is below different from the conventional 
seamless processing as a gestalt of the 3rd operation, composition performs 
lamination, shading-off processing is added along with the stuck boundary 
portion, and how to erase a knot is explained. 

[0157] In addition, only the contents of the synthetic processing which the 
configuration in the gestalt of this 3rd operation and actuation become 
unnecessary [ the judgment portion of the brightness histogram shown by 
drawing 29 ] although it is the same as that of the gestalt of the 1st operation 
fundamentally, and is shown by drawing 19 differ. Therefore, here explains 
only a different portion from the gestalt of the 1st operation. 
[0158] The panorama image composition means 517 performs shading-off 
processing by the gestalt of this operation, as shown in drawing 34 and 
drawing 35 . 

[0159] That is, with the gestalt of this operation, as shown in drawing 34 , 
the 1st image 3503 adds shading-off processing to a part for the edge 3501 as 
it is, using all as some synthetic images. Furthermore, as a filter of 
shading-off processing, it carries out using the matrix 3502 of 3x3 as shown, 
for example in drawing 35 . 

[0160] Next, the actuation in the gestalt of this operation is explained. 
[0161] The flow chart of the image composition processing in the gestalt of 
this operation to drawing 36 is shown. 

[0162] In this drawing, the panorama image composition means 517 secures 
a twice as many field as the 1st image (left image in drawing 34 ) as a 
synthetic image field first (S3401). And the 1st image is simply copied to this 
synthetic image field (S3402). 

[0163] next - the remaining fields (x y) of a synthetic image - formula 
x^Ax+By+Cy^-Bx+Ay+D - corresponding (x\ yO - it asks (S3403). And it 
confirms whether (x\ y f ) are in the 2nd image (right image in drawing 34 ) 
(S3404), and if there is nothing, processing will be advanced to the 
below-mentioned step S3402, and the pixel of if it is (x\ yO will be copied to (x, 
y) (S3405). 

[0164] Furthermore, if it judges whether processing from the above step 
S3403 to S3405 was repeatedly performed to all the remaining fields of a 
synthetic image (S3406) and there is no line, in order to repeat processing to 
all the remaining fields of a synthetic image, processing is returned to said 
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step S3403. 

[0165] Moreover, if it judges that it went by step S3406 to all fields, 
shading-off processing will be performed, covering the filter of 3502 to a part 
for the edge of the 1st image (3501 in drawing 34 ) (S3407), and processing 
will be ended now. 

[0166] Thus, although the conventional seamless processing is taking how 
composition performing lamination, adding shading-off processing along 
with the stuck portion as another different means, and erasing a knot and 
some knot remains with the gestalt of the 3rd operation compared with 
previous seamless processing, there is an advantage to which the processing 
time can be managed in a short time. Moreover, the effect of becoming 
unnecessary [ distinction called an alphabetic character subject's image and 
a natural image ] is acquired. 

[0167] (Gestalt of the 4th operation) Next, the gestalt of the 4th operation by 
this invention is explained. 

[0168] When printing a panorama image on below as a gestalt of the 4th 
operation later In the case of the printer (binary PURITA) by which it has a 
means by which a user specifies beforehand the printer which carries out a 
print, and the specified printer outputs binary [ which is printed by error 
diffusion methods, such as an ink jet printer (B J) and a laser beam printer 
(LBP), ] How to perform lamination of an image in Rhine (Rhine where 
brightness is high) where concentration is low is explained. 
[0169] in addition , although the configuration in the gestalt of this 4th 
operation and actuation be the same as that of the gestalt of the 1st 
operation fundamentally , when the binary printer print by the error 
diffusion method as a printer as mentioned above be specify , it be 
characterize by perform image composition bordering on Rhine where the 
brightness of the image in the duplicate image field be the highest , and the 
contents of the processing about this differ . Therefore, here explains only a 
different portion from the gestalt of the 1st operation. 

[0170] The flow chart which explains the whole actuation to drawing 37 is 
shown. 

[0171] First, the panorama image composition means 517 confirms whether 
the printer is assigned to the user (S3601), and it checks it in a binary 
printer (S3602). 

[0172] If it is a binary printer, processing explained henceforth will be 
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performed (S3603). The printer will not be specified, or if it is not a binary 
printer, the gestalt of the 1st operation will be processed (S3604). 
[0173] In addition, although synthetic processing with the gestalt of this 
operation is fundamentally the same as the gestalt of the 1st operation, 
distinction of the alphabetic character image shown in drawing 29 or a 
natural image becomes unnecessary. And it differs from the place in drawing 
19 which connects and asks for Rhine 3001. 

[0174] Drawing 33 is drawing also showing the flow chart of processing with 
the gestalt of this operation, and shows the processing actuation by the 
panorama image composition means 517 at the time of asking for the 
brightest Rhine from the range which the image overlaps. However, this flow 
chart explains as a case of composition on either side. Moreover, in the upper 
and lower sides, you may think the same way. 

[0175] First, the panorama image composition means 517 initializes the 
variable max holding maximum to 0 (S3301). And all the pixels belonging to 
vertical Rhine are acquired from the leftmost side of the overlap range 3002 
shown by drawing 19 (S3302). It substitutes for Variable Sum in quest of 
total of these pixel values (S3303). It judges whether Sum is larger than max 
(S3304), and if Sum is larger than max, it will be called brighter Rhine, and a 
value Sum will be assigned to max and will be held (S3305). Moreover, the 
location pos of Rhine is also held. When the above is performed to the 
rightmost side of an overlap range one by one, it will be called the brightest 
Rhine in the location pos currently held. Rhine of this location is set up as 
Rhine 3001 in drawing 19 . 

[0176] Next, the processing actuation at the time of performing actual 
synthetic processing by the panorama image composition means 517 is 
explained. Drawing 30 is drawing also showing the flow chart of processing 
with the gestalt of this operation. 

[0177] In this drawing, the panorama image composition means 517 secures 
a twice as many field as the 1st image (left image in drawing 22 ) as a 
synthetic image field first (S2201). And the field on the left of Rhine 3001 of 
the 1st image is simply copied to this synthetic image field (S2202). the 
remaining fields (x y) of a synthetic image - from formula 
x , =Ax+By+Cy--Bx+Ay+D - corresponding (x\ y f ) - it asks (S2203). 
[0178] Next, the pixel of if check (S2204) and it is [ whether (x\ y') are in the 
2nd image (right image in drawing 22 ) and ] (x\ yO is copied to (x, y) (S2205). 
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A repeat and processing end the above to all the remaining fields of a 
synthetic image. 

[0179] It has a means to specify the printer in the above cases of printing a 
synthetic image according to the gestalt of the 4th operation, and, in the case 
of the binary printer by which the printer specified as the user outputs BJ 
and binary [, such as LBP, ], Rhine to connect is made into Rhine (Rhine 
where concentration is the lowest) where brightness is the highest. Thereby, 
in the case of a binary printer, outputting by the error diffusion method is 
common, and since a printing dot diffuses the low place of concentration 
most at this time, the effect that a knot is not conspicuous is induced. 
Moreover, most printers in current office and a current home are binary 
printers, such as said BJ, LBP, etc., and it is thought further [ when / most / 
a user uses a printer clearly ] that this processing is applicable. 
[0180] 

[Effect of the Invention] By distinguishing whether it is the image with 
which the image of the image field which overlapped by having considered as 
the configuration mentioned above according to this invention makes an 
alphabetic character a subject, as explained in full detail above, and 
performing image composition, after performing a different image processing 
according to this distinction result, the problem that where of the knot in a 
panorama composition image will be conspicuous can be solved, and good 
image composition is attained. 

[0181] Moreover, although the resolution of a portion obscured by adding 
shading-off processing to the boundary portion which connects two or more 
images as another means different from the conventional seamless 
processing, and performing image composition falls a little, the high-speed 
processing in which a knot is not conspicuous is realizable. 
[0182] Furthermore, a printing dot diffuses the low place of concentration 
most in the case of a print, and, thereby, the effect that a knot is not 
conspicuous can acquire by performing image composition bordering on 
Rhine the brightness of the image in the duplicate image field is the highest 
in Rhine when the binary printer which prints by the error diffusion method 
as a printer which outputs a synthetic image is specified. 



DESCRIPTION OF DRAWINGS 



29/32 



Japanese Publication number : 09-093431 



[Brief Description of the Drawings] 

[Drawing ll It is the outline external view of this equipment. 
[Drawing 2] It is the block diagram showing the configuration of this 
equipment. 

[Drawing 3] It is drawing showing the case where the 1st example is applied 
in an alphabetic character image. 

[Drawing 4] It is drawing showing the histogram of the brightness in an 
alphabetic character image. 

[Drawing 5] It is drawing showing the histogram of the brightness in a 
natural image. 

[Drawing 6] It is drawing showing the data structure of the image data 
recorded in the electronic camera. 

[Drawing 7] It is drawing showing the screen at the time of copying the 
image data in an electronic camera. 

[Drawing 8] It is a data structure when managing data within a computer. 
[Drawing 9] It is drawing showing the combination of the image of two 
sheets assumed at the time of full auto composition processing. 
[Drawing 10] It is drawing showing the combination of the image of two 
sheets assumed at the time of full auto composition processing. 
[Drawing ll] It is drawing showing the combination of the image of two 
sheets assumed at the time of full auto composition processing. 
[Drawing 12] It is drawing showing the combination of the image of two 
sheets assumed at the time of full auto composition processing. 
[Drawing 13] It is drawing showing the user interface of auto composition 
processing. 

[Drawing 14] It is drawing showing the user interface of auto composition 
processing. 

[Drawing 15] It is drawing showing the user interface of semi auto 
composition processing. 

[Drawing 16] It is drawing showing the user interface of semi auto 
composition processing. 

[Drawing 17] It is drawing showing the matching range at the time of 
composition. 

[Drawing 18] It is drawing showing the template image and matching range 
of corresponding-points extract processing. 

[Drawing 19] They are an overlap range and drawing showing a setup of 
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Rhine to connect. 

[Drawing 20] It is drawing showing the histogram of the brightness for 
distinguishing an alphabetic character image or a natural image. 
[Drawing 21] It is drawing showing the histogram of the brightness for 
distinguishing an alphabetic character image or a natural image. 
[Drawing 22] It is drawing illustrating synthetic processing. 
[Drawing 23] It is the flow chart of the processing performed in case the 
image data in an electronic camera is copied. 

[Drawing 24] It is the flow chart which shows the flow in the whole case of 
panorama composition. 

[Drawing 25] It is the flow chart of auto composition processing. 
[Drawing 26] It is the flow chart of full auto composition processing. 
[Drawing 27] It is the flow chart of semi auto composition processing. 
[Drawing 28] It is the flow chart of corresponding-points extract processing. 
[Drawing 29] It is the flow chart which shows processing actuation of the 
whole including distinction of being an alphabetic character image. 
[Drawing 30] It is the flow chart of synthetic processing. 
[Drawing 31] It is a flow chart about the synthetic processing including 
seamless processing. 

[Drawing 32] It is drawing showing the case where the 2nd example is 
applied in an alphabetic character image. 

[Drawing 33] It is a flow chart at the time of asking for the brightest Rhine 
from an overlap range. 

[Drawing 34] It is drawing showing the shading-off processing in the 3rd 
example. 

[Drawing 35] It is drawing showing the filter used for shading-off processing. 
[Drawing 36] It is the flow chart of the synthetic processing in the 3rd 
example. 

[Drawing 37] It is the flow chart which shows processing actuation of the 4th 
whole example. 

[Drawing 38] It is drawing explaining seamless processing. 

[Drawing 39] It is drawing explaining seamless processing when overlap 

width of face is large. 

[Drawing 40] It is drawing showing the case where seamless processing is 
performed in an alphabetic character image. 
[Description of Notations] 
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301 Main Part of Computer 

302 Display 

303 Mouse 

306 General Interface 

307 Electronic Camera 



